This invention relates, in general, to processing streaming data, and, in particular, to improving processing of such data.
Processing within a data processing system can include various forms, including non-stream processing and stream processing. In non-stream processing, data is received, saved and later processed. In contrast, in stream processing, data is processed, as it is continuously received.
Examples of stream processing systems include large scale sense-and-respond systems, which continuously receive external signals in the form of one or more streams from multiple sources and employ analytics aimed at detecting critical conditions and, ideally, responding in a proactive fashion. Examples of such systems abound, ranging from systems deployed for monitoring and controlling manufacturing processes, power distribution systems, and telecommunication networks, to environmental monitoring systems, to algorithmic trading platforms, etc. These sense-and-respond systems share the need for:                (1) calculating baselines for multiple samples of incoming signals (e.g., instantaneous electricity production levels, the fair price of a security, among others); and/or        (2) correlating the computed value for a signal with other signals (e.g., instantaneous electricity consumption levels, the ask price of a security, among others).        
This paradigm of streaming analytics focuses on incremental processing as data is received from external sources. This differs from the typical store-and-process paradigm (e.g., non-stream processing) that answers queries by processing the needed data for that query at the time the query is issued. The advantage of incremental processing is the availability of analysis results with low latency and high throughput.